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Abstract 

Systematic evaluation of child and adolescent psychiatric outpatient treatment is important but time-consuming. 
The aim of this paper was to study whether Interactive Voice Response (IVR) is a more effective method than a 
questionnaire sent by post when following up outpatient treatment in child and adolescent psychiatry. Eighty 
patients were recruited from a child and adolescent psychiatric outpatient unit in Sweden. One parent of each of 
the patients was randomized to complete the BCFPI follow-up form, using either IVR (n = 40) or postal survey (n = 40) 
one month after discharge. The response rate for complete answers was 65% in the IVR group and 38% in the postal 
survey group (p = 0.014). There was less need for reminders in the IVR group (p = 0.000). IVR is a promising and 
cost-effective method for evaluating evidence-based treatment in child and adolescent psychiatric care. 

Keywords: Child and adolescent psychiatry; Evaluation; Interactive voice response (IVR); Postal survey; Brief child 
and family phone interview (BCFPI) 



Introduction 

Approximately 15% of all children and adolescents have 
a mental disorder that requires treatment (Blakemore 
2008). Common psychiatric syndromes are anxiety, de- 
pressive disorders and hyperactivity (Ries Merkangas 
et al. 2009), causing suffering for the young person in 
question as well as for their relatives and peers (Weisz 
et al. 2011). Evidence-based treatment options are avail- 
able for children and adolescents with psychiatric disor- 
ders (The Cochrane Collaboration 2013). The concept of 
evidence-based psychiatry does not only include the 
treatment: it also emphasises the importance of continu- 
ous and systematic evaluation of the treatment options 
to decide whether improvements are needed (Akobeng 
2005). In child and adolescent psychiatry there are 
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validated instruments to evaluate treatment (Dulcan 
2010). One example is the Brief Child and Family Phone 
Interview (BCFPI), which is a structured survey for clin- 
ical intake evaluation, treatment planning and follow-up 
assessment (Cunningham et al. 2009; Boyle et al. 2009). 
The interview involves parents, teachers, and adoles- 
cents. BCFPI is primarily designed as a phone interview 
performed by a clinical interviewer with formal training 
in children's mental health. According to the manual, 
the assessment takes approximately 30 minutes to 
complete (Cunningham et al. 2006), but our experience 
is that it takes longer, sometimes up to 60 minutes. The 
overall burden, not only on the departments but also on 
the individual clinicians, increases the risk that treat- 
ment evaluations will not be prioritized. To increase the 
probability of the evaluation being carried out, various 
kinds of automated technologies can be used to perform 
clinical evaluations. Interactive Voice Response (IVR) is 
an automated telephone system where a central com- 
puter is programmed to administer questionnaires; 
respondents answer pre-recorded questions by pressing 
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a number on the keypad. All answers are stored directly 
in the central computer. In a limited number of studies, 
IVR has been used to follow up adolescents in psychi- 
atric inpatient and outpatient care (Johansson et al. 
2013), but there are no previous studies comparing IVR 
and postal surveys when following up outpatient treat- 
ment in child and adolescent psychiatry. 

Aim 

The primary aim was to investigate whether IVR is a 
more effective method than postal surveys when collect- 
ing data for use in evaluating outpatient treatment in 
child and adolescent psychiatry. A secondary aim was to 
evaluate whether IVR requires less reminders compared 
to postal surveys. 

Methods 

About 250 000 children and adolescents live in the catch- 
ment area of the Department of Child and Adolescent 
Psychiatry, Malmo, Sweden. As part of ongoing develop- 
mental work, the department used the BCFPI parent 
interview as intake screening and as a follow-up instru- 
ment one month after discharge from treatment. In the 
current study a total of 80 parents to consecutive patients 
who had completed outpatient treatment during March- 
September 2008 were randomized to a BCFPI parent 
follow-up assessment with either IVR or a postal survey. 
For inclusion in the study, parents had to understand 
written and spoken Swedish. Those randomized to the 
IVR group (n = 40) received a letter informing them about 
the evaluation and giving a specific date and time for the 
automated phone interview. The IVR system was pro- 
grammed to make eight attempts during the evening if 
the parent did not respond. If the call was answered, a 
short introduction was given including a confirmation that 
the person answering was the one who completed the 
BCFPI intake interview, and then the BCFPI survey took 
place. Those randomized to the postal survey (n = 40) 
received a similar letter, but also including the BCFPI 
questionnaire. The parent was asked to complete the form 
and return it to the department using a pre-paid envelope. 
Regardless of randomization group, one reminder with 
the same procedure was sent by the department staff if 
the parent did not respond. 

Statistics 

Fisher's exact test was used to compare frequencies of 
response rates and reminders. Descriptive statistics are 
presented as mean (SD) for continuous variables and as 
counts for categorical data. All tests were two-tailed. A 
p-value of <0.05 was regarded significant (Altman 1990). 
SPSS version 20 was used for the statistical calculations. 



Results and discussion 

IVR proved to be more effective than postal surveys 
when using the BCFPI parent survey to follow up chil- 
dren and adolescents who had completed psychiatric 
outpatient treatment. In the IVR group, 26 parents 
responded to the automated phone interview, while 19 
responses were received from the postal survey group 
(p = 0.115). There was a higher proportion of complete 
answers in the IVR group (26/40, 65%), compared to the 
postal survey group (15/40, 38%) (p = 0.014). The ex- 
cluded surveys, i.e. insufficiently-completed BCFPI forms, 
could not be used to evaluate treatment. In the IVR group, 
17 parents received a reminder, compared to 36 parents in 
the postal survey group (p = 0.000), i.e. the department 
spent less time on sending reminders to participants in 
the IVR group. The mean (SD) time for the IVR re- 
spondents was 10:57 (1:40) minutes, compared with the 
30-60 minutes needed for a phone interview. 

There are several possible explanations for the higher 
rate of complete answers and for the reduced need for 
reminders in the IVR group. IVR requires an immediate 
answer, while the postal survey may easily be postponed 
or forgotten. Automated telephony is probably more 
challenging and offers less opportunity to omit certain 
items. The IVR responses are delivered instantly, while 
subjects in the other group have to put in additional 
effort to physically complete and then post the question- 
naire. The reason for the reduced time consumption for 
the respondents compared to that estimated for a clin- 
ical phone interview is probably that initial 'small talk' 
and comments between the answers in a person-to- 
person phone interview are omitted in the IVR proced- 
ure. The IVR superiority may increase the probability 
that treatment is actually evaluated, thereby releasing 
more time for clinical work for the trained clinicians. 

There are other advantages of using the IVR technol- 
ogy. The automated phone survey can be implemented 
without pre-training of the respondent, and data is 
stored directly in the central computer, allowing imme- 
diate analysis and feedback to clinicians. 

IVR reduces the need for clinicians to engage in admin- 
istrative tasks and check that data is collected, although 
data ultimately has to be interpreted and managed by 
clinicians. IVR uses validated instruments but in other 
areas, such as the diagnostic process, clinicians are still 
needed. When using validated instruments, it is important 
that questions are asked according to the manual, i.e. that 
data is not biased by the interviewer. IVR manages 
closed-response alternatives, including multiple-response 
options, e.g. patient medication, but can also administer 
qualitative, open questions by using recordings. Like pos- 
tal surveys, IVR has limited possibilities for additional 
follow-up questions as each question is pre-programmed, 
but both methods allow additional comments. 
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Strengths 

To our knowledge this is the first randomized study 
evaluating children and adolescents discharged from 
psychiatric outpatient treatment with IVR carrying a 
validated screening instrument. 

Limitations 

A limitation is that background data including diagnoses 
and treatment regimes, which could possibly explain 
some group differences, was not collected. Data on the 
amount of time needed for completing the paper survey 
was not obtained. A third study arm involving a person- 
to-person telephone BCFPI follow-up survey could have 
better answered the question of whether IVR follow-up 
is less time consuming. The study did not examine 
whether BCFPI follow-up with IVR and the paper survey 
have the same quality as an interview performed by a 
trained clinician. 

Future implications 

The IVR technology is easily implemented, and may be 
used to further evaluate and develop evidence-based 
treatment models in child and adolescent psychiatry. 
The IVR system could probably provide targeted feed- 
back to the parent based on the BCFPI follow-up result, 
e.g. congratulate a positive or stable result, or recom- 
mend support for those with a negative outcome. 

Conclusion 

IVR is a promising and cost-effective method offering 
high response rates and little burden on clinical staff and 
respondents. The technology is not personnel-dependent 
and could easily be implemented in child and adolescent 
psychiatric care to evaluate evidence-based treatment. 
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